<html>
<head><meta charset="utf-8"><title>Ask bors to verify that a test fails before a PR? · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html">Ask bors to verify that a test fails before a PR?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="206479983"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206479983" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206479983">(Aug 10 2020 at 16:15)</a>:</h4>
<p>A common issue I run into is that it's hard to tell whether the tests added in a PR are actually relevant. See e.g. <br>
<a href="https://github.com/rust-lang/rust/issues/65983#issuecomment-640256301">https://github.com/rust-lang/rust/issues/65983#issuecomment-640256301</a><br>
and <a href="https://github.com/rust-lang/rust/pull/75237#issuecomment-670788825">https://github.com/rust-lang/rust/pull/75237#issuecomment-670788825</a>.</p>
<p>Is it possible to have bors try to run those newly added tests on master and verify that they fail? That would give me a lot more confidence as a reviewer. It seems like it shouldn't impact CI times a ton since there are always artifacts already built for master.</p>



<a name="206480103"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206480103" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206480103">(Aug 10 2020 at 16:16)</a>:</h4>
<p>I guess this shouldn't <em>always</em> happen because sometimes you're just adding new tests with no code changes. So maybe it should be opt in like <code>@bors verify-tests</code></p>



<a name="206480793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206480793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206480793">(Aug 10 2020 at 16:22)</a>:</h4>
<p>I think putting this in bors is going to be essentially impossible, it's just not the right tool for the job</p>



<a name="206480823"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206480823" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206480823">(Aug 10 2020 at 16:22)</a>:</h4>
<p>you can of course first push up tests, edit the CI config for try to be a test-running builder, and run with that</p>



<a name="206480863"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206480863" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206480863">(Aug 10 2020 at 16:23)</a>:</h4>
<p>and we <em>could</em>, I guess, experiment with an allowed-to-fail builder that ran tests with downloaded binaries from master</p>



<a name="206480901"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206480901" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206480901">(Aug 10 2020 at 16:23)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F/near/206480823">said</a>:</p>
<blockquote>
<p>you can of course first push up tests, edit the CI config for try to be a test-running builder, and run with that</p>
</blockquote>
<p>that's a lot more work though haha</p>



<a name="206480929"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206480929" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206480929">(Aug 10 2020 at 16:23)</a>:</h4>
<p>you have to cherry-pick the files, make a PR, and link back to to the original</p>



<a name="206481005"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Ask%20bors%20to%20verify%20that%20a%20test%20fails%20before%20a%20PR%3F/near/206481005" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F.html#206481005">(Aug 10 2020 at 16:24)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/242791-t-infra/topic/Ask.20bors.20to.20verify.20that.20a.20test.20fails.20before.20a.20PR.3F/near/206480863">said</a>:</p>
<blockquote>
<p>and we <em>could</em>, I guess, experiment with an allowed-to-fail builder that ran tests with downloaded binaries from master</p>
</blockquote>
<p>that sounds really cool <span aria-label="heart" class="emoji emoji-2764" role="img" title="heart">:heart:</span></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>